Towards Sociable Virtual Humans: Multimodal Recognition of Human Input and Behavior

VIRSTORY: A Collaborative Virtual Storytelling

From the results of two Human behavior studies in small group interactions we constructed iteratively a Collaborative Virtual Environment named “VIRSTORY”. This system is a digital storytelling with speech and 3D gesture recognition technologies using like input devices. It includes several modules: multimodal interaction module, behavior module to animate autonomous expressive characters, etc.

Identity Emulation (IE): Bio-inspired Facial Expression Interfaces for Emotive Robots

Our facial expression robots use biomimetic structures, aesthetic design principles, and recent breakthroughs in elastomer material sciences to enact a sizable range of natural humanlike facial expressions. Applications of this class of human-robot interface devices will rise in relevance as humans and robots begin to have more sociable encounters in the coming years. The Identity Emulation pro...

Requirements and Building Blocks for Sociable Embodied Agents

To be sociable, embodied interactive agents like virtual characters or humanoid robots need to be able to engage in mutual coordination of behaviors, beliefs, and relationships with their human interlocutors. We argue that this requires them to be capable of flexible multimodal expressiveness, incremental perception of other’s behaviors, and the integration and interaction of these models in un...

At the Virtual Frontier: Introducing Gunslinger, a Multi-Character, Mixed-Reality, Story-Driven Experience

Gunslinger is an interactive-entertainment application of virtual humans that transforms an iconic Wild West movie scene into a vivid semblance of reality. The project combines virtual humans technology with Hollywood storytelling and set building into an engaging, mixed-reality, story-driven experience, where a single participant can interact verbally and non-verbally with multiple virtual cha...

Behavior Generation for Interpersonal Coordination with Virtual Humans: on Specifying, Scheduling and Realizing Multimodal Virtual Human Behavior